Adjusting effective multiplicity (Meff) for family-wise error rate in functional near-infrared spectroscopy data with a small sample size

Abstract. Significance The advancement of multichannel functional near-infrared spectroscopy (fNIRS) has enabled measurements across a wide range of brain regions. This increase in multiplicity necessitates the control of family-wise errors in statistical hypothesis testing. To address this issue, the effective multiplicity (Meff) method designed for channel-wise analysis, which considers the correlation between fNIRS channels, was developed. However, this method loses reliability when the sample size is smaller than the number of channels, leading to a rank deficiency in the eigenvalues of the correlation matrix and hindering the accuracy of Meff calculations. Aim We aimed to reevaluate the effectiveness of the Meff method for fNIRS data with a small sample size. Approach In experiment 1, we used resampling simulations to explore the relationship between sample size and Meff values. Based on these results, experiment 2 employed a typical exponential model to investigate whether valid Meff could be predicted from a small sample size. Results Experiment 1 revealed that the Meff values were underestimated when the sample size was smaller than the number of channels. However, an exponential pattern was observed. Subsequently, in experiment 2, we found that valid Meff values can be derived from sample sizes of 30 to 40 in datasets with 44 and 52 channels using a typical exponential model. Conclusions The findings from these two experiments indicate the potential for the effective application of Meff correction in fNIRS studies with sample sizes smaller than the number of channels.


Introduction
In the resampling and predictive simulations conducted in Experiment 2 of the main article, it was suggested that the prediction of valid M eff is feasible when the sample size (N) is more than 60 to 70% of the number of functional near-infrared spectroscopy (fNIRS) channels (M).Specifically, it was observed that the prediction error remained below or around 1 (in multiplicity) or 5% when N ranged from 30 to 40 in datasets with 44 or 52 channels.
However, when N constitutes a low percentage of M, the standard deviation (SD) of the prediction increased considerably.This elevates the risk of either insufficient correction due to underestimation or excessive correction due to overestimation of Meff.In fNIRS analysis, obtaining a sufficient N against M for an accurate prediction can sometimes be difficult.This issue becomes particularly notable in multi-channel measurements utilizing several dozen to over 100 channels.To minimize the prediction error using the exponential model, an adequate N is necessary.However, if achieving this is difficult, the applicability of the M eff correction should also be assessed.
In such cases, a modification through penalties can prevent the risk of insufficient correction by accounting for the underestimation of M eff while accepting the risk of conservative correction.In this supplementary material, we examine the possibility of applying penalties based on the ratio of N to M through the results of Experiment 2. We will introduce a function, Penalty = f (N/M), designed to compute the penalty value from the N to M ratio.

Calculation of penalty value
To derive the function, we utilized the SD of the ratio between the target M eff value and the predicted M eff value for each N.The target value is denoted as   , and the predicted value is    (i ranging from 1 to 1000).This ratio ( , ) can be interpreted as an indicator of prediction accuracy.
A  , greater than 1 indicates an underestimation by the prediction, whereas a value less than 1 suggests an overestimation.Thus, the SD of  , reflects the variability in prediction accuracy for each N of the simulations in Experiment 2.
It is important to note that, here, the penalty was calculated by dividing target values by predicted values.In actual experiments, the target values are unknown.In this simulation, however, both target and predicted values could be obtained.The penalty was calculated based on the predicted value.

Derivation of functions
The  , for each N was calculated 1000 times using four data sets, which are described in the main article, to establish the function, Penalty = f (N/M).Subsequently, the   ,• for each N was calculated.
Penalty values obtained from the four datasets were combined to plot the relationship between N and M. From the approximate curve, Penalty = f (N/M) was derived.The function curve enabled the calculation of penalty values for a specific M and N within any given dataset.

Calculation of double-adjusted Meff
The double-adjusted M eff value can be calculated by applying the derived penalty value as follows:

Results
We plotted the penalty values obtained from the four datasets against each N/M ratio with an exponential approximation applied.The root mean square error (RMSE) exhibited a small value at 0.037, below 0.1.This analysis led to the derivation of a formula to calculate the penalty based on the Fig. 1 The blue dots represent penalty values computed across four datasets, and the red line is an exponential approximation.

Practical examples
Here, we will describe practical examples of the computation of double-adjusted M eff using Go/No-go task data, which is described in the main text.In the predictive simulations of Experiment 2, the average, minus SD, of predicted values represents potentially underestimated predictions in actual analyses.We calculated the double-adjusted M eff assuming that this value (the average, minus SD, of that the need for a penalty value has decreased.On the other hand, the penalty value increased when N/M ratios were below this point, leading to conservative corrections similar to the Bonferroni method.
Therefore, to leverage the advantages of the Meff correction method, it is desirable to obtain as large a N as possible.
material, we have proposed the modification of predicted M eff values by introducing a penalty value when N is below the threshold recommended for accurate prediction.From the simulation results of Experiment 2, a function penalty = f (N/M) was derived to calculate the penalty value based on the ratio of N to M. This modification through penalty values can prevent underestimation of predicted M eff values obtained from a dataset with a small N.In the practical examples, underestimated M eff values were adjusted to be closer to the target values.The graph of penalty = f (N/M) (Fig. 1) shows that the penalty value drops below 0.1 when N/M is about 0.5.When N exceeds this point, the value added by the penalty approaches the SD of target values, suggesting